R Notebook

Applied exercises

This exercise relates to the College data set, which can be found in the file College.csv. It contains a number of variables for 777 different universities and colleges in the US. The variables are:

• Private : Public/private indicator • Apps : Number of applications received • Accept : Number of applicants accepted • Enroll : Number of new students enrolled • Top10perc : New students from top 10 % of high school class • Top25perc : New students from top 25 % of high school class • F.Undergrad : Number of full-time undergraduates • P.Undergrad : Number of part-time undergraduates • Outstate : Out-of-state tuition • Room.Board : Room and board costs • Books : Estimated book costs • Personal : Estimated personal spending • PhD : Percent of faculty with Ph.D.’s • Terminal : Percent of faculty with terminal degree • S.F.Ratio : Student/faculty ratio • perc.alumni : Percent of alumni who donate • Expend : Instructional expenditure per student • Grad.Rate : Graduation rate Before reading the data into R, it can be viewed in Excel or a text editor.

1. Use the read.csv() function to read the data into R. Call the loaded data college. Make sure that you have the directory set to the correct location for the data.

2. Look at the data using the fix() function. You should notice that the first column is just the name of each university. We don’t really want R to treat this as data. However, it may be handy to have these names for later. Try the following commands:

In the package version of College the row names are already assigned as the unviersity names.

1. Use the summary() function to produce a numerical summary of the variables in the data set.
2. Use the pairs() function to produce a scatterplot matrix of the first ten columns or variables of the data. Recall that you can reference the first ten columns of a matrix A using A[,1:10].
3. Use the plot() function to produce side-by-side boxplots of Outstate versus Private.
4. Create a new qualitative variable, called Elite, by binning the Top10perc variable. We are going to divide universities into two groups based on whether or not the proportion of students coming from the top 10% of their high school classes exceeds 50 %.

library(ggplot2)
lapply(College, summary)

pairs(College[1:10])

boxplot(College$Outstate ~ College$Private)

College$Elite <- ifelse(College$Top10perc > 50, "Yes", "No")

Use the summary() function to see how many elite univer- sities there are. Now use the plot() function to produce side-by-side boxplots of Outstate versus Elite.

22. Use the hist() function to produce some histograms with differing numbers of bins for a few of the quantitative vari- ables. You may find the command par(mfrow=c(2,2)) useful: it will divide the print window into four regions so that four plots can be made simultaneously. Modifying the arguments to this function will divide the screen in other ways.

hist(College$Enroll)

hist(College$F.Undergrad)

hist(College$PhD)

hist(College$Accept/College$Apps) # Per every application, barely anyone accepts a student.

hist(College$Grad.Rate[College$Private == "Yes"])

hist(College$Grad.Rate[College$Private == "No"])

6. Continue exploring the data, and provide a brief summary of what you discover. To do

9. This exercise involves the Auto data set studied in the lab. Make sure that the missing values have been removed from the data.

1. Which of the predictors are quantitative, and which are quali- tative?

lapply(Auto[-ncol(Auto)], summary)

$mpg
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   9.00   17.00   22.75   23.45   29.00   46.60 

$cylinders
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  3.000   4.000   4.000   5.472   8.000   8.000 

$displacement
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   68.0   105.0   151.0   194.4   275.8   455.0 

$horsepower
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   46.0    75.0    93.5   104.5   126.0   230.0 

$weight
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1613    2225    2804    2978    3615    5140 

$acceleration
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   8.00   13.78   15.50   15.54   17.02   24.80 

$year
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  70.00   73.00   76.00   75.98   79.00   82.00 

$origin
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  1.000   1.000   1.000   1.577   2.000   3.000 

2. What is the range of each quantitative predictor? You can answer this using the range() function.

lapply(Auto[c("mpg", "displacement", "horsepower", "weight", "acceleration", "year")], range)

$mpg
[1]  9.0 46.6

$displacement
[1]  68 455

$horsepower
[1]  46 230

$weight
[1] 1613 5140

$acceleration
[1]  8.0 24.8

$year
[1] 70 82

3. What is the mean and standard deviation of each quantitative predictor?

lapply(Auto[c("mpg", "displacement", "horsepower", "weight", "acceleration", "year")], mean)

$mpg
[1] 23.44592

$displacement
[1] 194.412

$horsepower
[1] 104.4694

$weight
[1] 2977.584

$acceleration
[1] 15.54133

$year
[1] 75.97959

4. Now remove the 10th through 85th observations. What is the range, mean, and standard deviation of each predictor in the subset of the data that remains?

basic <- function(x) {
  summary <- c(mean = mean(x, na.rm = T), range = range(x, na.rm = T), sd = sd(x, na.rm = T))
  summary
}

lapply(Auto[-(10:85), -ncol(Auto)], basic)

5. Using the full data set, investigate the predictors graphically, using scatterplots or other tools of your choice. Create some plots highlighting the relationships among the predictors. Comment on your findings.

It’s interesting to look the mpg is negatively related to displacement, horsepower and weight in a similar fashion. In fact, the correlation are:

correlations <- c(first = cor(Auto$mpg, Auto$displacement), second = cor(Auto$mpg, Auto$horsepower), third = cor(Auto$mpg, Auto$weight))
correlations

     first     second      third 
-0.8051269 -0.7784268 -0.8322442 

All within a 5 figure difference. and above -0.70. In contrast, the correlations between displacement, horsepower and weight are strong and positive.

correlations2 <- c(displa_horse = cor(Auto$displacement, Auto$horsepower), displace_weight = cor(Auto$displacement, Auto$weight), horsep_weight = cor(Auto$horsepower, Auto$weight))
correlations2

   displa_horse displace_weight   horsep_weight 
      0.8972570       0.9329944       0.8645377 

plot(Auto$displacement, Auto$horsepower)

plot(Auto$displacement, Auto$weight)

plot(Auto$horsepower, Auto$weight)

6. Suppose that we wish to predict gas mileage (mpg) on the basis of the other variables. Do your plots suggest that any of the other variables might be useful in predicting mpg? Justify your answer.

Yeah, just as we saw in the matrix scatterplot, mpg is highly correlated with several variables. Not only visually, but the correlation coefficients are substantial.

1. To begin, load in the Boston data set.

How many rows are in this data set? How many columns? What do the rows and columns represent?

dim(Boston)

[1] 506  14

The data has 506 rows and 14 columns. Each row represents housing values in suburbs from Boston. See help(Boston).

2. Make some pairwise scatterplots of the predictors (columns) in this data set. Describe your findings.

plot(Boston)

Well, besides the array of correlations that you can find in the scatterplot, I find interesting that there’s some clear non-linear patterns such as in nox ~ dis as well as Istat ~ medv and rm ~ Istat.

3. Are any of the predictors associated with per capita crime rate? If so, explain the relationship.

crim is correlated with age, div, lstat and medv. The relationships are not straightforward though.

As the proportion of owner-occupied units buitl prior to 1940 increases, crime rates increase. But note that there’s threshold effects. At around 40% crime rates start to increase and only around 80% it becomes substantial. This relationship might not be that strong linearly, but more non-linear.

With dis (distance to five Boston employment centres), crime rates are higher only when employment centers are close, but as it passes increases over 4, crime rates set at 0.

With lstat it’s similar as with age, crime rates increases only after some value, more or less around 15. It seems that crime rates are not linearly related to almost any variable; it’s mostly non-linearly related. Same pattern with medv

4. Do any of the suburbs of Boston appear to have particularly high crime rates? Tax rates? Pupil-teacher ratios? Comment on the range of each predictor.

Boxplot(Boston$crim)

 [1] 381 419 406 411 415 405 399 428 414 418

Above the graph we can see the outliers which have significantly higher crime rates than the overall distribution. 10 suburbs, so not specifically a small bunch.

Boxplot(Boston$ptratio)

 [1] 197 198 199 258 259 260 261 262 263 264

Boxplot(Boston$tax)

ptratio does have some outliers, but they’re not the same ones as with crime rates. Surprisingly, tax doesn`t have any outliers, and the distribution is pretty smooth.

for (i in c("crim", "ptratio", "tax")) {
  print(range(Boston[, i]))
}

[1]  0.00632 88.97620
[1] 12.6 22.0
[1] 187 711

The range of crime rates can be as high as 88% and as low as 0%, quite some diversity between suburbs. Student-teacher ratio is much smaller, with a maximum of 22 students and a minimum of 13. Tax ranges from 187 to 711.

5. How many of the suburbs in this data set bound the Charles river?

table(Boston$chas)

  0   1 
471  35 

35 suburs.

6. What is the median pupil-teacher ratio among the towns in this data set?

median(Boston$ptratio)

[1] 19.05

19 pupils per teacher.

7. Which suburb of Boston has lowest median value of owner- occupied homes? What are the values of the other predictors for that suburb, and how do those values compare to the overall ranges for those predictors? Comment on your findings.

Boston[Boston$medv == min(Boston$medv), ]

Actually, both suburb 399 and 406 have the same lowest media value, which is 5.

First, both suburs have virtually the same values for all predictors except for the crim, lstat and black.

8. In this data set, how many of the suburbs average more than seven rooms per dwelling? More than eight rooms per dwelling? Comment on the suburbs that average more than eight rooms per dwelling.